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Abstract 

In  this  paper  we  describe  two  engineering  experiments 
designed  to  evaluate  the  effectiveness  of  Measurand’s 
ShapeTape  for  wearable,  mobile  interaction.  Our  initial  re¬ 
sults  suggest  that  the  ShapeTape  is  not  appropriate  for  inter¬ 
actions  which  require  a  high  degree  of  accuracy.  However, 
ShapeTape  is  capable  of  reproducing  the  qualitative  motion 
the  user  is  performing  and  thus  could  be  used  to  support  3D 
gesture-based  interaction. 

1.  Introduction 

The  advent  of  wearable  computing  has  created  the  need  for 
new  human-to-computer  interaction  paradigms  and  inter¬ 
face  devices.  While  keyboards  and  mice  are  sufficient  for 
desktop  computers,  they  are  not  appropriate  for  wearable 
computers  where  more  natural  3D  interactions  are  required. 
We  postulate  that  a  successful  hand  tracking  system  for  a 
wearable,  mobile  system  needs  to  be  body-mounted,  ac¬ 
curate  enough  to  support  the  desired  interactions,  is  easy 
to  configure  and  calibrate,  and  does  not  interfere  with  the 
user’s  normal  body  interactions.  Potential  technologies  in¬ 
clude  computer  vision  [1],  ultrasonics  [2]  and  Measurand 
Corporation’s  ShapeTape  [3].  ShapeTape  is  a  thin  flexible 
ribbon  composed  of  bend  and  twist  sensors  which  measure 
the  attitude  of  the  tape  at  various  positions  along  its  length. 
Given  the  fact  that  it  is  a  small,  lightweight,  low-powered 
technology  which  does  not  rely  on  extensive  signal  process¬ 
ing  and  does  not  restrict  the  user’s  movements,  ShapeTape 
appears  to  be  well-suited  for  the  problem  of  3D  interac¬ 
tion  and  gesture  recognition.  To  assess  its  suitability,  we 
integrated  it  into  the  Battlefield  Augmented  Reality  System 
(BARS)  [4]  and  performed  two  engineering  tests. 

2.  System  Setup 

The  ShapeTape  was  integrated  into  the  BARS  hardware.  As 
shown  in  Figure  1,  the  tape  controller  was  mounted  horizon¬ 
tally  on  a  metal  back  pack  frame  (silver  rectangle  visible  on 
the  left  hand  side  of  the  picture).  The  tape  was  laid  along 
the  top  of  the  user’s  right  arm  and  wrapped  around  the  index 


Figure  1 :  Virtual  Tape  (Light)  Overlaid  on  Top  of  the  Real 
Tape  (Dark) 

finger  of  the  user’s  right  hand.  The  excess  tape  was  looped 
in  the  user’s  palm. 

The  ShapeTape  API  divides  the  tape  into  a  set  of  seg¬ 
ments.  The  attitude  of  each  segment  is  calculated  directly. 
The  calibration  process  consisted  of  calculating  fixed  off¬ 
sets  for  each  of  these  measurements.  The  tape  was  held  hor¬ 
izontally  and  the  offsets  were  calculated  directly.  To  assess 
the  accuracy  of  the  calibration,  we  used  the  BARS  system 
to  synthesise  a  virtual  tape  and  overlaid  it  on  a  video  of  the 
real  world  which  was  captured  by  a  calibrated  video  cam¬ 
era.  Figure  1  shows  the  typical  results  of  the  calibration  ex¬ 
ercise  and  reveals  several  issues  with  the  ShapeTape.  First, 
because  the  position  of  a  segment  s  is  found  by  integrating 
all  previous  measurements,  errors  accumulate.  In  particu¬ 
lar,  errors  near  the  base  of  the  tape  lead  to  very  large  errors 
in  its  end.  Second,  the  calibration  tends  to  drift  over  time. 
Therefore,  it  was  necessary  to  perform  the  calibration  pro¬ 
cedure  multiple  times  within  any  experimental  run.  Third, 
despite  careful  calibration,  we  were  not  able  to  achieve  very 
accurate  tracking  results.  Figure  1  shows  the  accuracies  still 
present  even  after  the  calibration  process. 

3.  Experiments 

3.1.  Angular  Pointing 

We  first  tested  the  performance  of  ShapeTape  as  a  pointing 
device.  A  two-dimensional  grid  of  9  x  9  points  was  pre¬ 
cisely  drawn  on  a  white  board.  The  grid  was  positioned  and 
centered  to  create  a  field  of  view  of  45  degrees  horizontally 
and  vertically  for  the  user.  The  ShapeTape  controller  was 
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Figure  2:  User  Performing  Pointing  Operations  in  Pointing 
Experiment 
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Figure  3:  Overhead  View  of  the  Pointing  Experiment 

placed  at  a  known  fixed  position  corresponding  to  the  lo¬ 
cation  on  the  user’s  back.  This  eliminated  errors  from  the 
body  tracker.  As  shown  in  Figure  2,  the  experiment  required 
the  user  to  point  at  each  grid  sample  using  their  tracked  fin¬ 
ger.  An  overhead  view  of  the  user  pointing  at  the  grid  points 
is  shown  in  Figure  3. 

The  position  of  the  pointer  for  each  point  of  the  grid  was 
recorded  as  well  as  the  position  of  the  user’s  eye  during  the 
pointing.  The  position  of  the  user’s  eye  was  determined 
from  a  head-tracked  helmet.  Azimuth  and  elevation  point¬ 
ing  errors  were  calculated  from  the  angular  difference  be¬ 
tween  the  line  going  through  the  user’s  eye  and  the  position 
of  the  point  on  the  grid  and  the  line  going  through  the  user’s 
eye  and  the  measured  position  of  the  tip  of  the  ShapeTape. 
We  found  that  angular  pointing  errors  increased  as  the  user 
deviated  from  the  calibrated  position  and  could  be  as  high 
as  10°.  This  suggests  that  the  angular  resolution  of  object 
placement  for  a  given  environment  should  not  be  smaller 
than  10°  if  one  wants  to  select  objects  without  errors.  This 
granularity  is  too  high  for  our  application. 

3.2.  Gesture  Recognition 

Although  it  is  not  accurate  enough  for  unstructured  point¬ 
ing  gestures,  ShapeTape  might  be  sufficiently  accurate  for 


gesture-based  interaction  where  absolute  accuracy  is  of  lim¬ 
ited  importance.  Therefore,  we  investigated  the  use  of 
ShapeTape  as  a  method  for  entering  text.  The  position  of 
the  tape’s  end  segment  was  recorded  and,  after  projecting 
the  points  onto  a  two-dimensional  plane,  the  results  were 
sent  to  a  Graffiti  gesture  recognizer  [5] .  The  user  interface 
toolkit  SATIN  [6]  was  used  to  perform  this  task. 

Several  users  were  able  to  successfully  enter  text  into 
BARS.  However,  users  indicated  that  keeping  their  arm  high 
up  in  order  to  draw  the  string  was  not  comfortable,  and  as 
a  consequence  their  gestures  were  inexact  with  respect  to 
what  they  wanted  to  draw. 

4.  Summary  and  Conclusions 

We  investigated  the  appropriateness  of  Measurand’s 
ShapeTape  device  for  wearable  mobile  interaction.  We  per¬ 
formed  two  experiments:  one  to  assess  the  absolute  pointing 
accuracy  of  the  tape,  the  other  to  assess  whether  it  can  be 
used  in  a  gesture-based  recognition  system.  Pointing  errors 
were  found  to  be  unacceptably  large,  and  could  be  as  high 
as  10°.  However,  it  was  accurate  enough  for  users  to  enter 
text  using  a  Graffiti-based  gesture  recognizer.  Therefore, 
we  conclude  that  ShapeTape  is  not  appropriate  for  interac¬ 
tions  which  require  a  high  degree  of  accuracy.  However, 
ShapeTape  is  capable  of  reproducing  the  qualitative  motion 
the  user  is  performing  and  thus  could  be  used  to  support  3D 
interaction. 
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